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Introduction 


Touch screen is a screen display that responds and is sensitive to the touch of a finger or stylus and provides di- 
rect interaction with information. It is most widely used in automatic teller machines (ATM), interactive displays 
in retail environments, GPS systems, computers, and recently in mobile phones. Penetration of touch screen 
mobile phones has made a significant jump, especially after the introduction of Apple's iPhone. Touch screens can 
be easy to use with almost no training or less training if an application is designed appropriately. 


There has been development of touch screen in almost all possible areas of mobile application, health care, 
railway and bus route information application, museum, touch screen devices such as city guides, office environ- 
ments, educational applications, industrial control panels, entertainment sector, gaming, computers and laptops, 
household appliances and many more. 


In the current scenario, due to technological advancement and ease of use, a wide variety of touch screen devic- 
es are available such as mobile touch devices, tablets, interactive kiosks, and large wall-mounted, free-standing 
and encased touch screen devices. Here, we introduce you to touch screen technologies, finger and touch target 
consideration, patterns and gestures used to make touch screen interfaces and design considerations of design- 
ing touch screen interfaces. 
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Infrared Touch Screen 


An array of infrared Light Emitting Diodes (LEDs) and photo detector pairs are placed around the edges of the 
panel. The LEDs are placed both horizontally and vertically creating a grid of infrared light beams. When a finger 
or any object touches the screen, the grids of LED beams are obstructed, helping to find out the X and Y position- 
ing of the object approaching the touch screen. 


Infrared touch screen can detect any object (e.g. finger, stylus, glove etc.) approaching the touch screen, which 
makes it a desirable technology to use. Additionally, it can be scaled to any size without losing the resolution. 
Light transmission is also considered better than resistive touch screens. 


Front view of Infrared (IR) Touch Screen 
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Designing Touch Screen Gestures 


The figure above showcases the front view of infrared touch screen. 
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Designing Touch Strnen Gestures 
Figure above showcases the touch event caused in infrared touch screen. 
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Resistive Touch Screen 


Resistive touch screen is made of two most important conductive layers. These layers are electrically conductive 
and separated from air gaps or microdots. When a finger or any other object (e.g. stylus) touches the outer layer 
of touch screen, the connecting points of two layers act as a voltage divider. In simpler language, when a finger 
or any other object (e.g. stylus etc.) touches the outer layer of the touch screen, there is a notified change in elec- 
tric current at the connecting point of two conductive layers, which eventually is registered as a touch point. 


The two conductive sheets have horizontal and vertical lines. When finger or any other object touches the outer 


layer of the touch screen, the lines help detect the x and y position, eventually helping register the precise loca- 
tion of touch. 


Resistive Touch Screen 
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Figure showcasing the internal structure of resistive touch screen. 
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Capacitive Touch Screen 


In a capacitive touch screen panel, a conductive layer is placed on the top of glass layer of the monitor. The con- 
ductive layer is made up of transparent capacitive material (e.g. indium tin oxide-ITO). A small voltage is applied 
to the conductive layer that resulting in a uniform electrostatic field. In a simpler language, an electric charge is 
stored on the conductive layer. When a human finger touches the touch screen, some of the electric charges are 
transferred to the user, which decreases the charge on the conductive layer. The decrease in the charge is meas- 
ured by circuits located at each corner of the touch screen panel. A relative difference in electric charge is calcu- 
lated, showcasing where the touch event took place. This information is passed on to the computer software and 
appropriate action is taken to accomplish the task of the user. 


Capacitive touch screens are dependent upon conductive properties of an object; so human finger is easily de- 
tected. Normal gloves and stylus does not work with capacitive touch screen due to their non-conductive proper- 
ties. Capacitive touch screen does not rely on the pressure given to the surface, which makes it more responsive 
to relatively complex gestures such as pinch & shrink to zoom out, pinch & spread to zoom in, slide, swipe etc. 


Capacitive Touch Screen 


Small voltage is applied from 
every cornor of the touch screen panel 


Electric charge is stored 
on the touch screen panel 


Finger touches the screen, 
causing change in electric 
charge. The alectric charge change 
is calculated to kdentify where 
the touch event took place 


Figure showcasing the working of capacitive touch screen. 
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Figure showcasing the internal structure surface acoustic wave based touch screen. 
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Figure showcasing the operation when a finger approaches SAW based touch screen. 
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Wii Based Touch Screen 


Wii remote console based touch screen was developed by Johnny Lee. Wiimote is a bluetooth enabled gesture 
based remote control that is made available with Nintendo Wii gaming console. To make a touch screen, Wii 
remote console is sufficient instead of a complete gaming console. 


To make a Wiimote based touch screen: 


* A button enabled infrared (IR) light pen, 
* A Wii remote console and 
¢ A software (developed by Johnny Lee) is required. 


Wiimote detects the infrared light attached to pen. When the button of the pen is pressed, software detects the 
positioning of the pen on the screen and sends a signal to the software; causing it a touch event. The detection 
can also be done by keeping the pen away from the screen. It does not require touching the IR light pen on the 
screen. The touch screen size can be scaled from a normal laptop screen to a large projected screen. 


Wii based touch screen is relatively cheaper than other touch screens, though it requires a small set up to be 
made to make Wii based touch screen. Wii based touch screen works well with rear projection. 


More details can be found at Johnny Lee’s website: http://johnnylee.net/projects/wii/ 
Wii remote console can be bought in India in the range of INR 1600-2000. 
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Finger and Touch Targets 


Selecting Finger Size: 

Diameter of an adult finger is typically of 16mm to 20mm. Generally, for pushing buttons and using touch screen, 
finger pad is used. They are narrower than full finger but wider than the fingertip. The size of the finger pad is 
10-14mm. 


Image of a finger pad. Image of a fingertip. 


Fingertip is narrower than finger pad and full finger. The size of the fingertip is 8-10mm. 
Calculating Target Size: 
The interface element where user is asked to touch to select, should not be smaller than 1cm diameter or 


Icm*icm 


° “Size of the target element = Target size in inches (0.4 inch) * Pixel per inch” 
* “Pixel per inch (PPI) = Screen width in pixel/screen width in inch” 


The higher the PPI, the larger your interface elements will have to be to create suitable touch targets. 
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Touch Targets 


Generally, the target size of the control is the same as the visible control. 
E.g. If the button size is 100 (width) X 40 (height) (pixel), the target size of the button is kept same as 100 (width) 
X 40 (height) (pixel). 


Though in some cases, there are possible variations based on the screen size, application content and context of 
use. There are majorly two ways around the size limitations of touch targets. 


Iceberg Tips: 

Iceberg tips are the controls that have larger targets than visible ones. The touch target size can be increased 
without increasing the size of the element itself. Sometimes as designer, you may not want to element size to in- 
crease from a specific size, and then iceberg tips can be helpful. There has to be enough space between elements 
to create this effect. 


Submit 


+ # 


Iceberg Tips. 
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Gestures of Touch Screen 


Patterns for touch screens described here, in some cases can also be used for free form interactive gestures. 


These examples showcase a set of touch gestures, though these are not meant to be the only set of touch ges- 
tures. The touch gesture may vary and heavily dependent upon context of use, type of application, touch screen 
technology etc. 


Tap to open/activate/select: 
Tapping on a specific area or specific interface objects will open or activate the element. Whenever there is a 
need for a target object to be activated or a button to be pushed to active, tap to open/activate can be used. 


In addition to open/activate, tap can be used to select as well. Here, one tap selects an element and another tap 
opens/activates. Tap to select is used where user taps on a target object to select for manipulation. Also, when 
user needs to select multiple options from list of choices, tap to select can be used. 


Tap is a simplest touch gesture and is a better replacement for left mouse click. As mouse over event occurs in 
pointer based interactions, there is less possibility to trigger the same event in touch based interactions. 


There are two types of taps to open/activate: 


* Open/ Activate/ Select on press 
* Open/ Activate/ Select on release 


Most of the mechanical buttons use activate on press method, though in case of touch screens, it is better to 
use open/activate/select on release method as it gives user time to move off the button if he changes his mind, 
especially if tapping performed to an action that cannot be undone. 
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1. Introduction When an action takes longer time to perform or user wants to interrupt during ongoing performance, especially 


if the ongoing action is displaying a set of screens or contents that user may want to act on it, tap to stop can 
2. Types of Touch Screen be used. When a particular action is performed, tapping can be used to stop the performance of the action. e.g. 
3. Finger and Touch Targets tapping on the screen to stop the scroll. 
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Tap to stop can cancel/stop the action, or pause the action depending upon the need. 
Tap to Stop 
a 
ee ee 


Tap to Stop. 
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Double Tap: 
In few handheld devices, double is tap is used to select an object and perform action associated with that object. 
When user needs to choose from multiple options associated with the selected object, double tap used. 


First tap on the screen selects the object, second tap can be either choosing from multiple options or viewing the 
details of selected object. 


Double Tap (On Press) Double Tap (On Release) 


Carpe Poet over Gamers 


Double Tap On Press. 


Carpe Fo een Geer 


Double Tap On Release. 


Drag and Drop: 

Drag and drop is basically placing a finger on the top of an object and moving it along towards a new location, 
with the help of a finger. Users need a gesture to move objects from one place to another, so pushing and pulling 
an object to different place is a natural gesture to move them. 

It can be used when object needs to be placed at different locations by user, for scrolling purposes, sliders etc. 


In drag and drop, object needs to be tapped and then pushed or pulled to another location. Drag and drop can 
also be combined with tap to select, where object is selected first and then moved it across the screen. Drag and 
drop can also be constrained to be moved in a particular direction. E.g. moving an element just on left side or 
right side. Similar way elements can be dragged only horizontally, not vertically. Use shorter drags, users may not 
be able to understand longer drags and may cause accidental drops. 
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6. Touch Feedback 

7. Design Consideration Flick: 

8. Contact Details If a set of contents, a screen needs to move away, flick gesture can be used. A slight flick by index finger in any 


direction can be used to flip through screens, move away contents and help quickly flip through contents. Here, 
the index finger slightly touches the screen and moves in a straight line in one direction. If a tip of a finger is 
used, it can be considered as a flick and if the finger pad is used, which relatively takes more time on screen, can 
be considered as push or pull. Based on the speed of the flick, the movement of the object will continue after the 
gesture is complete, slowing to a gentle stop. 
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7. Design Consideration Touchdown and Drag: 


8. Contact Details When contents such as a number of files, folders, text etc. needs to be selected together on a screen, needs to 
place at different location or perform any action, touchdown and drag is used. It is similar to dragging a mouse 
around the computer desktop to select target objects. 


Index finger is used to touch the screen (where object is not present), slides the finger around the selected area 
of the screen. A visual feedback is essential to display the selected area on the screen and the selected objects. 
Further, necessary actions can be performed on the selected objects depending on the objects. E.g. a number of 
images can be selected to touchdown and drag and together they can be moved to trash. 
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7. Design Consideration Touchdown and Hold: 

8. Contact Details When a number of actions are associated with the selected object, and needs to be shown to user to perform 


any of those actions, touchdown and hold gesture is used. It is similar to a right click on mouse based interaction. 


An index finger is placed on the target object and kept on hold until the information is displayed, a set of actions 
are showcased to the user. User can choose the appropriate action to complete the task. It is recommended to 
showcase the set of action close to the fingertip, so action can be chosen easily. For example, touchdown and 
hold can be used to quickly edit images on a touch screen interface. User can choose target image, hold it and set 
of actions such as crop, blur, sharpen etc. can be showcased to the user for image editing. 
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8. Contact Details Many screens have a smaller visible area, specifically mobile devices, where all the contents are not possible to 


show at one go. Slide to scroll is used when user wants to see the contents that resides outside the visible screen 
area. For example, viewing a map, list of items that go beyond the visible screen area such as phonebook contact, 
messages etc. 


Though this is similar to drag and drop movement, but specific to sliding a finger on the screen in one direction, 
that helps scroll the screen or list of the items in one direction. Slide to scroll should not be used to with two 
fingers to scroll for clarity purpose. 
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Slide to Scroll 


Gesigning Towch Screen Gestures 
Slide to Scroll. 


Slide and Hold for Continuous Scroll: 
One rotation on the scroll can go through the full list of items on the screen. A speed or time threshold can also 
be set so that after it is reached the scrolling can speed up. 


Rapid scrolling can be difficult for users to follow, so it is best if the user understands the structure of the list. 
(e.g. list can be alphabetical order) It can also be difficult to precisely stop on a particular item. User often goes 
back and forth to select the chosen item. Scroll bar indicating position can also be helpful; either making it 
always visible to making it visible whenever necessary. Visual and haptic feedback can be helpful to follow the 
scrolling of the items. A visual feedback is also helpful such as haptic feedback, visual bounce or sound. Also it is 
helpful to indicate the user when he reaches to end of the scroll. 
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Slide to Hold for Continuous Scroll 


Cesigning Touch Screen Gestures 
Slide and Hold for Continuous Scroll. 


Spin to Scroll: 

In addition to slide to scroll, spin to scroll is used to view the contents that are not visible on the screen due to 
smaller screen size. Typically a user moves his thumb or index finger in a circular motion to scroll. A clockwise 
gesture scrolls down/forward/right and a counterclockwise gesture scrolls up/backward/left. Spin to scroll can 
with used on touch screen as well as on a circular trackpad such as an iPod. Spin to scroll allows for rapid scrolling 
with a motion that is more continuous and smoother than up and down or side to side scrolling. One rotation 

on the scroll can go through the full list of items on the screen. A speed or time threshold can also be set so that 
after it is reached the scrolling can speed up. Rapid scrolling can be difficult for users to follow, so it is best if the 
user understands the structure of the list. (e.g. list can be in alphabetical order) 


It can also be difficult to precisely stop on a particular item. The user often goes back and forth to select the 
chosen item. A scroll bar indicating position can also be helpful; either making is always visible to make it visible 
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whenever necessary. Visual and haptic feedback can be helpful to follow the scrolling of the items. 


Spin to Scroll 


Besigning Touch Screen Gestures 
Spin to Scroll. 


Fling to scroll: 

In simple words, fling to scroll is a powerful version of flick. Fling to scroll is a strong fling of index finger in any 
direction that moves the screen in that direction. Fling to scroll is used to rapidly go through the large number of 
screens, contents or text. Flick and fling to scroll are natural interaction to move around the screen contents that 
mimics the physical world. The tip of the index finger (though it can be other fingers as well) touches the screen 
and moves in a straight line in one direction. The rate of the fling is translated into momentum, which is slowed 
and eventually stopped by simulated friction. Feedback on the gesture is very important. Feedback can be a 
visual bounce, haptic, sound etc. 
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Fling to Scroll 


EH 


Cesigning Touch Screen Gestures 


Fling to Scroll. 


Two fingers to scroll: 

In addition to slide to scroll, drag and drop, there is another way to go through a large amount of content or 
screens. Two fingers, typically an index finger and a middle finger are used to scroll through screens, a large 
amount of text, contents etc. Two fingers can be placed anywhere on the screen and then typically moved 
straight in one direction to go upward/downward/right/left. 


Two fingers to scroll are generally used in trackpad (e.g. trackpad used in Apple MacBook series). Two fingers to 
scroll require less attention to scroll through contents as it can be placed anywhere on the screen, while slide to 
scroll requires conscious attention on the slider to scroll. This gesture allows control of number of elements while 
retaining control of individual objects via cursor. (In case of trackpad) To accommodate this gesture, the surface 
needs to be sensitive to two touches on the surface. 
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Daaapniney Touch Screen Gattures 


Two Fingers to Scroll. 


Push button to scroll: 

In addition to drag and drop, flick and fling to scroll, another way of scrolling through large number of infor- 
mation can be a simple push button. A simple push button can be placed at the bottom of the screen, tapping 
it whenever user needs to scroll. A long time can do the scrolling faster based on time duration the button is 
pressed. 


Push Button to Seroll 


Keegy Teac Eevee: fore 


Push button to Scroll. 
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Pinch to shrink and spread to enlarge: 

To check out every detail on a small touch screen device, users need to zoom in and zoom out the contents to 
check the details. When a document size, photograph, maps etc. need to be increased or decreased, pinch to 
shrink and spread to enlarge gesture is used. Two fingers, either the thumb and index finger of the same hand 
or index finger of both hands are used and they are brought closer together (pinch to shrink) or further apart 
(spread to enlarge) while on the top of the object, they decrease and increase the size respectively. 


Pinch to shrink and spread to enlarge are generally found together. These two gestures require multi-touch capa- 
bility as system should be able to detect two touch events on the screen. Typically elements those sizes are to be 
increased and decreased are scaled proportionally. The starting points of two fingers are usually equal to 100% 
of the object’s starting size. As fingers move closer together or further apart, the element scales in real time to 
match the new distance between two fingers at a ratio that designers need to determine based on the sizes of 
the original elements and the size of the screen. e.g. if the fingers are 50% closer, the element can be 50% of it 
original size. Designers need to create a point beyond that users cannot shrink or enlarge the object. 


Finch to Shrink Spread to Enlarge 


Ca agnieg Touch Scroan Gealurie 


Pinch to Shrink. 


Ce agnieg Touch Scrogn Gesunde 


Spread to Enlarge. 
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Depgning Touch Screen Gestures 
Swipe. 
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Ghost Fingers: 
Ghost fingers are visualization of user’s fingers when the fingers are out of users’ line of sight such as on the oth- 
er side of a mobile device, inside an object or on the other side of a wall. 


When the user has to see her fingers to manipulate controls but prevent them from seeing because of the hard- 
ware and environment, ghost fingers are used. 


Through ghost fingers you may be able to overcome the limitation of opaque surfaces and screen coverage, place 
controls in more ergonomic way and in places that make more sense. If the back of the device can be used, this 
gesture allows the use of all 10 fingers at once. 


Ghost finger gesture requires sensors to detect the positioning of hands. A second touch pad or additional cam- 
eras can be added to detect the positioning of the fingers. Shadow like fingers can be used for the visualization 
of ghost fingers. 


Ghost Fingers 


Ghost Fingers. 
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Free Form Gestures: 

In touch screen interfaces, freeform gestures can be used to upgrade personalized security measures for users. 
For example, a personalized freeform gesture can be incorporated to unlock the keypad and further operations 
for use in a touch based mobile device. 


Users should be given a freedom to choose the personalized gesture, so they can better remember it. It is very 
essential to understand the negative side of providing freeform gestures. Proper care should be taken while giv- 
ing freedom to choose a personalized gesture and there must be a way out if user forgets the gesture. 


There are endless possibilities to use freeform gesture depending on the context of use, application area and 
content. 


Other Touch Screen Gestures: 

There can be a number of gestures added depending on the application & functionality of use. 
For example: 

- Making a cross (x) gesture to delete the content/ information. 

- Identifying users with their gestures. 

- Horizontal scrolling to move across number of screens and many more can be thought of. 


Though while defining new gesture, 


- It should be kept in mind to keep them as natural as possible. 
- People should be able to relate the gesture with the action performed and result obtained. 


Tap to Select 
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Touch Feedback 


It is very essential for users to know whether the action has been successfully performed. A good feedback meth- 
od can make touch interfaces easy to understand, learn and operate. 


Types of feedback: 

There are 3 types of feedback relating to touch screens: 
* Visual feedback 

* Sound feedback 

* Haptic feedback 


Visual feedback: 
It includes visual bounce, color changes when action is performed, screen is moved left/ right/ up/ down, target 
element glows when touched, 3D motion of performed action, hourglass display to showcase waiting period etc. 


Sound feedback: 
When the user touches the target element, several sound feedbacks can be provided. For example, click sound, 
beep sound, actions can be spoken by system etc. 


Haptic feedback: 
A vibration effect can be provided to every touch done by a user. This will indicate user about his performed 
action. 
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Design Consideration 


« Represent the objects in recognizable way. 
¢ Design for direct manipulation with the objects on screen. Reduce the number of objects kept hidden. 


* Nails do not work for fewer touch screens, such as capacitive touch screens. Though they are more accurate 
than finger pad and fingertip, but in some touch screens, they may not trigger the touch event. Appropriate care 
should be taken while choosing the type of touch screen, especially when the target users are female. 


* Typing on touch screen can be tedious and straining for wrist, especially on vertica mounted touch screens for 
a longer period of use. The arrangement of a keyboard may make the task of typing easier. 


¢ Also, the height of vertically mounted, wall mounted and large size touch screens should be properly chosen to 
make the system easy to use. Users should not struggle (bend too much or jump) to reach the information. 


¢ Human palm is accidentally touched to the screen, activating other elements on the interface. One solution is 
to make a vertical arrangement for the touch screen, which may work for interactive kiosks, but not for mobile 
phones. 


¢ In an open environment used touch screens (e.g. used for city guides, bus information system at bus stops 
etc.) the screen get dirty due to the environment. Appropriate care should be taken to choose the type of touch 
screen, casing if required, and placement of contents /actionable objects. 


¢ During the touch screen use, the hand covers to significant space of the screen, which is not visible. Try not to 
put instructions, labels, and contents below the interface element that can be touched. 


* Mouse based gestures may not be suitable for touch interfaces. Natural gestures are best way to think of using 
new gestures for touch interfaces. 


¢ There are environmental challenges while designing touch screen interfaces. In cold weather, people tend to 
wear gloves, which may require interface modifications and type of touch screen to be used. 


* The aim of touch screen interfaces should be for direct manipulation. This makes interface direct and easy to 
understand. If haptic feedback is not possible to accommodate, try using several iterations of visual and sound 
feedback. (e.g. visual bounce, color change, increase and decrease icon size, animation of one screen moving to 
another screen, beep sound etc.). 
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* There is no hover in touch-based interfaces. It is a good idea to tell people how to touch and where to touch. 
Another way of looking at it is, actionable objects can be designed (size, color, shape etc.) to attract users to 
touch. 


* The touch screen interface aims at direct manipulation. Try to avoid hidden contents. 
¢ It can be beneficial to use a grid structure to divide the screen and provide content appropriately. E.g. the 
content structure on screen can be divided into display elements, input elements, keyboard section, and status 


display. 


* Location of the structured content grid is beneficial to keep consistent for easy usage. Similar way, touch feed- 
back can also be made consistent throughout the interface. This will help users to perceive them as unity. 


e Limit the number of total screens to a smaller number. 


¢ Input data entry such as a keyboard or numeric entry can be made available whenever required. It needs not to 
be all the time on the screen, especially for a longer period of use. 


* It is recommended to use graphical buttons (of course, that should look like a button) to perform an action in- 
stead of using a text-based link. Though if the screen space is relatively smaller and completely character based, 
then text-based links can be used. 


* It is recommended not to show cursor on the touch screen interfaces. 
¢ There are two ways to have numeric entry arrangement. 
¢ Telephone layout 


* Keyboard layout 


Telephone layout can be useful for users as they can relate the numeric entry with other existing telephones. 
Especially when numeric entry is designed for calling purposes. 


Keyboard layout can be useful for user who is known to use keyboard and its numeric layout, though it may not 
be a good idea to accommodate the same calling purpose. 


Positively, It provides an opportunity for designers to brainstorm natural gestures that can be used for touch- 
based systems. If the target size of the element is small, finger precision can be a worry and be less accurate. 


D’source 


Digital Learning Environment for Design - w 


Design Course 


Touch Screen Gestures 
Touch Screen Technologies 

by 

Prof. Keyur Sorathia 

DoD, IIT Guwahati 


Source: 
https://www.dsource.in/course/touch-screen- 
gestures/contact-details 


1. Introduction 

2. Types of Touch Screen 

3. Finger and Touch Targets 
4. Touch Targets 

5. Gestures of Touch Screen 
6. Touch Feedback 

7. Design Consideration 

8. Contact Details 


Contact Details 


This documentation for the course was done by Prof. Keyur Sorathia at DoD, IIT Guwahati. 


You can get in touch with him at keyur[at]iitg.ernet.in or keyurbsorathia[at]gmail.com 
More details about his research can be found on www.embeddedinteractions.com 
You can write to the following address regarding suggestions and clarifications: 


Helpdesk Details: 

Co-ordinator 

Project e-kalpa 

Department of Design 

Indian Institute of Technology Guwahati 
Guwahati 781039 

Assam, India 


Phone: +91-361-2582500/ +91-361-2582451 
Fax: +91-361-2690762 
Email: dsource.in[at]gmail.com 
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